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At sufficiently low temperatures, the configurational phase space of a large spin-glass system 
breaks into many separated domains, each of which is referred to as a macroscopic state. The system 
is able to visit all spin configurations of the same macroscopic state, while it can not spontaneously 
. jump between two different macroscopic states. Ergodicity of the whole configurational phase space 

of the system, however, can be recovered if a temperature-annealing process is repeated an infinite 
^ ' number of times. In a heating-annealing cycle, the environmental temperature is first elevated 

to a high level and then decreased extremely slowly until a final low temperature T is reached. 
Different macroscopic states may be reached in different rounds of the annealing experiment; while 
\Q ' the probability of finding the system in macroscopic state a decreases exponentially with the free 

energy F a (T) of this state. For finite-connectivity spin glass systems, we use this free energy 
Boltzmann distribution to formulate the cavity approach of Mezard and Parisi [Eur. Phys. J. B 20, 
217 (2001)] in a slightly different form. For the ±J spin-glass model on a random regular graph of 
degree K = 6, the predictions of the present work agree with earlier simulational and theoretical 
CZ5 . results. 
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INTRODUCTION 



Spin-glasses are simple models for disordered systems. They can be defined very easily in mathematical terms; 
on the hand, the properties of such simple models usually are quite rich. Statistical physics of spin-glasses has been 
studied for more than thirty years since the concept of spin-glasses was first presented by Edwards and Anderson 
[l[ in 1975, but there are still many unsolved and heavily debated issues. In the last decades there have been a lot 
of theoretical investigations concerning models defined on a finite-connectivity random graph. These later models 
are more realistic than conventional spin-glass models (e.g., the Sherrington-Kirkpatrick model Q) on a complete 
graph, in the sense that each spin interacts only with a finite number of other spins. As direct analytical studies of 
spin-glass models on three-dimensional (regular) lattices are still beyond reach, people hope that a deep understanding 
of (mean-field) models on finite-connectivity random graphs will shed much light on the properties of 3D systems. 

For a spin-glass system of very large size, it is generally believed that ergodicity is broken when the environmental 
temperature T becomes lower than a certain value T sg , the spin-glass transition temperature. In this low-temperature 
spin-glass phase, the time average of a given physical quantity no longer equals to the ensemble average. The whole 
configurational phase space of the system breaks into many separated domains. Ergodicity is still preserved within 
each such configurational domains. If the system initially is in a spin configuration that belongs to domain a of 
the configurational space, it will eventually visit all other configurations in this domain as time elapses. On the 
other hand, due to the existence of very high free energy barriers, the system is unable to transit spontaneously 
from one configurational space domain to another different domain. In this ergodicity-broken situation, each such 
configurational phase space domain is regarded as a macroscopic state or thermodynamic state. The free energy F a 
of a macroscopic state a is related to the microscopic configurations in a through the following fundamental formula 
of statistical mechanics, 

F a (N,p) = -T\nZ a (N,p) , (1) 

where N is the total number of spins in the system; (3 = 1/T is the inverse temperature (we set Boltzmann's constant 
to unity throughout this paper); and Z a (N,/3) is the partition function for the macroscopic state a as defined by 

Z Q (JV,/3) = £exp [-/?#(<?)] . (2) 



* Citation information: Frontiers of Physics in China 2: 238-250 (2007). 
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In Eq. @, a denotes a microscopic spin configuration, and H(a) is the total energy of this configuration. The 
summation in Eq. |(5J) is over all those microscopic configurations belonging to macroscopic state a. 

For a given spin-glass system, although we can formally write down the expressions of the partition function 
and free energy for a macroscopic state a, the principal difficulty of spin-glass statistical physics is that we do 
not know a priori how the configurational space of the system is organized and which are the constituent spin 
configurations of each macroscopic state. To overcome this difficulty, one possibility is to first assume certain structural 
organization of the system's configurational space and then try to derive a self-consistent theory. For the Sherrington- 
Kirkpatrick model, the full-step replica-symmetry-broken (FRSB) theory of Parisi with an ultrametric organization 
of macroscopic states Q has met with great success. For finite-connectivity mean-field spin-glasses, a cavity approach 
was also developed by Mezard and Parisi [i| . This cavity approach combined the Bethe-Peierls approximation [E [E 0] 
for a ferromagnetic Ising model with the physical picture that there is a proliferation of macroscopic states in a 
spin-glass system. This approach, which has been shown [i| to be equivalent to the first-step replica-symmetry- 
broken (1RSB) replica theory, can give very good predictions concerning the low-temperature free energy density 
of a system on a random graph. Later on, this cavity approach was extended by Mezard and Parisi [8[ to the 
limiting case of zero temperature. The zero-temperature cavity method was applied to some hard combinatorial 
optimization problems with good performances (see, e.g., Refs. [E [l(J HH, G3 ) • These interdisciplinary applications 
in return also call for further understanding on the mean-field cavity approach and its possible extensions (see, e.g., 

Refs. pi M EE EE EE El). 

For a spin-glass system at a low temperature T, the total number of macroscopic states with a given free energy 
F is denoted as £l metc (F). Although it is natural to anticipate that the logarithm of fl mac (F) should be an extensive 
quantity, the exact relationship between 17 mac and F is unknown and is system-dependent. Inspired by Parisi's FRSB 
solution, in the cavity approach of Mezard and Parisi [4( one assumes that the total number £l mei c(F) of macroscopic 
states with free energy F diverges exponentially with F with respect to a reference free energy F r [3(, i.e., 

^mac (-^) 

oc exp(x0(F - F r fj , (3) 

where < x < 1 is a dimensionless constant to be determined self-consistently. The Parisi parameter x stems from the 
replica theory of infinite-connectivity spin-glasses 0], its physical meaning in the cavity framework is not transparent. 
For some spin-glass systems with many-body interactions, even if the exponential form of Eq. ^ is valid in certain 
range of free energy values, the parameter x in this equation may exceed unity. The original cavity iterative equations 
of Ref. [i| diverge for this situation of a; > 1, where the statistical physical property of the spin-glass system is not 
determined by those macroscopic states with the globally minimal free energy density. Since the cavity approach 
of Mezard and Parisi is a very useful theoretical tool in studying the low-temperature properties of many spin-glass 
systems, it might be helpful for us to interpret the approach from an another slightly different angle. Complementary 
interpretations of the 1RSB cavity theory will also facilitate its further development. 

In this paper we demonstrate that, the 1RSB cavity formalism of Mezard and Parisi can be derived in an alternative 
way without using Eq. ([3]). This slightly revised cavity theory is based on the gedanken experiment of repeated 
temperature heating-annealing. The final macroscopic state reached at the end of an annealing process is anticipated 
to follow the Boltzmann distribution of free energies. This theoretical approach is applied to the ± J spin-glass model 
on a random regular graph of degree K = 6 to test its validity. The results of the present work are in close agreement 
with earlier simulational and numerical results. The mathematical format of the present cavity theory is the same 
both for non-zero temperatures and for zero temperature. If the macroscopic states of a spin-glass system further 
organize into clusters of macroscopic states, it can be easily extended to take into account this situation. 

This paper is organized as follows. The next section introduces the ±J spin-glass model and the ensemble of random 
regular graphs of degree K . Section IIIII describes the free energy Boltzmann distribution of macroscopic states. In 
Sec. [TV] the concepts of cavity field and cavity magnetization are re-introduced, and the distribution of a vertex's 
cavity magnetization among all the macroscopic states is calculated. Various thermodynamic quantities, including 
the grand free energy density of the whole system and the free energy density of a macroscopic state, are calculated 
in Sec. IVl and Sec. I VII for an ensemble of systems and for a single system, respectively. The numerical results for the 
± J spin-glass model are reported and analyzed in Sec. IVIII We conclude the present work and discuss its possible 
extensions in Sec. IVIII1 The appendix gives an explicit expression for the mean energy density. 

II. THE ±J SPIN-GLASS MODEL ON A RANDOM REGULAR GRAPH 

In this paper we focus on just a single example, the ± J spin-glass model [l9j on a random regular graph of degree K . 
This model was also studied in Ref. [J], lending us the opportunity to directly compare the results of both treatments. 

Let us consider the graph Gk(N) which is obtained by randomly choosing with equal probability a graph from the 
set of all regular graphs of size N and vertex-degree K. Qk{N) is a random regular graph of degree K. Each of the N 
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vertices of Gk{N) is connected to K other vertices, but there is no structure in the connection pattern of Gk(N). For 
each edge of graph Gk(N), we assign a coupling constant = +J or Jy = — J with equal probability. Once 
the coupling for an edge is assigned, it no longer changes. Therefore, we have a network with random quenched 
connection pattern and random quenched coupling constants. (In the remaining part of this paper, when we use the 
term 'network', we mean the connection pattern of the graph plus the quenched couplings; when we use the term 
'graph', we mean only the connection pattern.) On top of such a network we define the following energy function 

H(<Ti, (72, ■ ■ ■ , CTN', G) = - JijCTiO-j , (4) 

(i,j)ee 

where 0$ (i = 1, . . . , N) is the spin variable of vertex i, a% can take two values, <7j = — 1 and <n = +1. Equation Q 
is the ±J spin-glass model. The total number of microscopic configurations a = {o~x, o~2, ■ ■ ■ , cjv} in such a system is 
simply 2 N . 

The vertex degree of a random regular graph of degree K is equal to that of a regular square lattice in d = K/2 
dimensions. One hopes that some statistical physical properties of a spin-glass model on a random regular graph will 
also hold for the same model on a regular lattice. On the other hand, in a regular lattice, there exist many short 
loops, which make analytical calculations extremely difficult. In a random regular graph of very large size there is no 
such short loops. The graph is locally tree-like; and the typical loop length in the graph scales as \og K _ 1 (N). One 
can exploit this absence of short loops to construct a self-consistent mean-field theory. 

For the benefit of later discussions, we also mention the concept of a random regular cavity graph GK(N,m) jij. 
In this graph of size N, there are m (m <C N) vertices of degree K — 1 and N — m vertices of degree K . A vertex of 
degree K — 1 in graph Gk{N, to) is referred to as a cavity vertex. Like Gk(N), the connection pattern of Gk(N, to) is 
also completely random; and the quenched coupling constants for the edges of Gk (N, to) are also independently and 
equally distributed over ± J. The spin-glass Hamiltonian Eq. ([4]) can also be defined for this cavity network. 



III. THE GEDANKEN ANNEALING EXPERIMENT AND THE BOLTZMANN DISTRIBUTION OF 

FREE ENERGIES 



Consider the spin-glass system Eq. @. At high temperatures, the system is in the paramagnetic phase. Each 
vertex of the network does not have any spontaneous magnetization, its spin fluctuates over the positive and negative 
directions and stays in each orientation with equal probability. The configurational phase space of the system is 
crgodic (see left panel of Fig. [T]). When the temperature T is decreased below a spin-glass transition temperature T sg , 
the system is in the spin-glass phase. In this phase, a vertex of the system might favor one spin orientation over the 
opposite orientation, but this orientation preference is vertex-dependent. When the number N of vertices is sufficiently 
large, the configurational phase space of this spin-glass system splits into many separated domains (see right panel 
of Fig. [I}. Each domain of the phase space, which corresponds to a macroscopic state or thermodynamic state of the 
system, contains a set of microscopic spin configurations. The system is ergodic within each macroscopic state, while 
ergodicity is broken at the level of macroscopic states. However, the system is able to transit from one macroscopic 
state to another different macroscopic state if the following temperature heating-annealing process is performed: The 
system is first heated to a high temperature beyond T sg and waited for a long time till it reaches equilibrium. The 
system now stays in a high-temperature ergodic phase. Afterwords, it is cooled infinitely slowly till the final low 
temperature T is reached. (A simulated-annealing idea was previously explored by Kirkpatrick and co-workers (20| 
to tackle hard optimization problems.) Since in the high-temperature phase the system loses memory about its prior 
history, at the end of the annealing process it may reach any a macroscopic state. The probability P a of the system 
end up being in a particular macroscopic state a, however, is in general different for different macroscopic states. We 
argue in the following that, if an infinite number of the above-mentioned annealing experiment are performed, P a 
should depend only on the free energy F a of the macroscopic state a. 

The total partition function of the system is defined as 

Z(N,0) = 5>xp (-/?#(<?)) , (5) 

a 

where H(a) is the total energy expression as given by Eq. When ergodicity is broken, the partition function 
Z{N, j3) can be re- written as a sum over all the macroscopic states a: 



z(N,p)=Y,J2 e M-P H (°)) = J2 z «( N >fi = J2 e M-i 3F <*( N 'p) 

a tr <E ct ct a 



(6) 
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FIG. 1: A schematic picture of ergodicity breaking in a spin-glass system. (Left) When the temperature is high, all the 
microscopic configurations (denoted by dots) which contribute to the free energy of the system are in a single domain of the 
configurational phase space. (Right) when the temperature is low enough, the microscopic configurations which contribute to 
the free energy of the system are contained in different domains in the configurational phase space. Each domain contains a 
number of microscopic configurations. Ergodicity is still preserved within each phase space domain, while the configuration of 
the system is unable to jump spontaneously from one domain to another different domain of the phase space. 



In Eq. ©, Z a {N,(3) and F a (N,/3) are the partition function and free energy of the macroscopic state a as defined 
in Eq. @ and Eq. ([TJ) , respectively. By way of repeated temperature annealing, the whole configurational space 
of a spin-glass system is explored (and ergodicity is recovered!). The total partition function Eq. ([5]) can therefore 
be understood as containing all the information of the system as measured by an infinite number of temperature 
annealing experiments. From Eq. © we see that each macroscopic state a contributes a term e~P Fo, ( N -^ to the total 
partition function of the system. Therefore wc can anticipate that, at temperature T, the probability of the system 
being in macroscopic state a at the end of an annealing experiment is given by the following distribution 

e -pF a {N.,f3) 

This is a free energy Boltzmann distribution over all the macroscopic states a. 

At this point, we introduce an artificial inverse temperature y at the level of macroscopic states. (Such an artificial 
inverse temperature y was first introduced in the early work of Ref. [|[ for T = spin-glasses.) This is better explained 
by constructing the following artificial single-particle system: the particle has a set of 'energy' levels, the 'energy' of 
level a is equal to the free energy F a (N 7 (3) of the macroscopic state a of the actual spin-glass system; this one-particle 
system is in a heat bath with inverse temperature y. Then the partition function of the artificial system is 

Z(N,P;y) = Y / ^p(-yF a (N,p)Y (8) 
The probability of this artificial system being in energy level a is given by 

When the inverse temperature y of the artificial system is set to y = {3, then the partition function Z[N, /3; y) reduces 
to the total partition function 2(N, (3) of the original spin-glass system. 

The partition function Z(N,(3;y) of Eq. ((5J) can be written in another form as 



Z(N,f3;y) = ^exp(-y^ Q ) =^n mac (F)e^ F = £ exp 



N(X(f)-yf) 



(10) 



In Eq. (|10p . / = F/N is the free energy density of a macroscopic state; the function £(/) is called the complexity 
0; 9 1 which is related to the total number f2 mac (F) of macroscopic states by the following equation 

E(/) = ilnfi mBC (JV/). (11) 
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The complexity £(/), which should be non-negative, is a measure of the total number of macroscopic states with free 
energy density /. When the spin-glass system is very large (N ^> 1), Eq. (fTQ|) indicates that the partition function 
Z(N, P) y) is contributed exclusively by those macroscopic states whose free energy density / satisfying the equation 



dm) 

df 



y 



(12) 



Equation (|12[) gives an implicit relationship between the inverse temperature y and the observed mean free energy 
density / of the system. 

For the benefit of later discussions, we define a grand free energy G(N, (3; /) by the following equation 



G(N, (3; y) = - - In Z(N, 0; y) = - - In 
V V 

From Eq. (JTUJ) we see that the grand free energy density g(f3; y) is 



g{P; y) = Jim 



G(N,P;y) 
N 



mm 

/ 



(/ -E(/)/y) = fifty) -E(mv))/y 



(13) 



(14) 



where f{(3;y) is the solution of Eq. (fT2"]) . 

In the next three sections we will study the statistical physical properties of the ± J spin-glass system using (3 
and y as a pair of control parameters. The free energy at the microscopic level (within a macroscopic state) will 
be calculated with the inverse temperature (3, while the grand free energy at the macroscopic state level will be 
calculated with the inverse temperature y. Although in the actual spin- glass system, the two inverse temperatures 
are identical (y = (3), they are decoupled in the following analytical theory. This decoupling gives us extra freedom in 
the theoretical development. Finally, to go from the artificial system to the original system, we will choose the largest 
value of y in the interval of < y < (3 which satisfies the requirement that the complexity X is non-negative. 

To analytically study the statistical physical property of a spin-glass model on a random graph, there are basically 
two different but equivalent approaches. The first one is the replica method [l|| [U and the second one is the 
cavity method [H|. The present paper exploits the cavity approach. It corresponds to the 1RSB approximation of the 
replica method. Basically, one assumes that macroscopic states of the spin-glass system arc distributed evenly in the 
whole configurational phase space, and there is no further organization of the macroscopic states or further structures 
within each macroscopic state. 



IV. INTEGRATING THE BOLTZMANN DISTRIBUTION OF FREE ENERGIES INTO THE CAVITY 

APPROACH 



A. Cavity fields 

Consider the spin-glass system Eq. Q on a cavity network Qk(N,K — 1) of N vertices and K — 1 cavity vertices 
(see Fig. UK)- Let us suppose that the configurations of such a cavity system are in a macroscopic state a. 

At given inverse temperature (3, the spin value Oj of a cavity vertex j fluctuates over time around certain mean 
value rrij which depends on the macroscopic state a. Since Uj is a binary variable, the marginal distribution Pj{o~j) 
of tjj can be expressed in the following form 



the magnetization of the cavity vertex j is 



^ )= c^V (15) 



rrij — tanh [3hj . (16) 



From Eq. (|16j) we know that hj is the magnetic field experienced by the cavity vertex j due to the spin-spin interactions 
between vertex j and its if —1 nearest- neighbors. We call hj the cavity field on vertex j and rrij the cavity magnetization 
of vertex j. We emphasize again that hj and rrij depends on the macroscopic state a. 

Since the connection pattern in a cavity graph with K — 1 cavity vertices is completely random, the typical length 
d(i,j) of a shortest-distance path between two cavity vertices j and k in the cavity graph Qk(N,K — 1) is a large 
value. Actually, it can easily be shown [23| that this distance scales logarithmically with the cavity graph size N: 

d(j,k) ~ In N. (17) 
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FIG. 2: Cavity vertices in a random regular cavity graph of vertex degree K (here K = 4). (A) The neighborhoods for three 
cavity vertices j, k and I. These three vertices have vertex degree K — 1 (=3) while all the other N — 3 vertices in the graph 
have vertex degree K. In a macroscopic state a, the three cavity vertices feel the cavity fields hj, and hi, respectively. (B) 
A new cavity graph with N + 1 vertices is generated by adding a new vertex i and connecting it to K — 1 old cavity vertices. 
The cavity vertex i of the new graph feels the cavity field hi in the corresponding macroscopic state a of the new cavity system. 

Denote Pj,k,..X a j> u k<, ■ ■ ■) as the joint probability distribution of the spin values <7j, <7fc, ... of a group of cavity vertices 
in a cavity network. In a large random graph, according to Eq. (|17p the shortest path length between any two 
randomly chosen vertices is long. Therefore, as the zeroth-order approximation, one may assume that this joint 
probability distribution can be written as the following factorized form 

Pi,fc,...(cr 7 -,o'fc,...) = pj{aj)pk{(Tk) • > ■ , (18) 

where pj [<jj ) is vertex £s marginal spin value distribution as given by Eq. (|15D . Equation (|18p is called the Bethe- 
Peierls approximation [E [E 0] in the literature. It assumes statistical independence among the spin states of the 
cavity vertices within a macroscopic state a. (For recent references on extensions of the Bethe-Peierls approximation, 
see Refs. HE EE E3, El-) 

B. Cavity field distribution among different macroscopic states 

The cavity magnetization nij of the cavity vertex j depends on the identity of the macroscopic state a. Its value 
may be different in different macroscopic states. Let us denote Vj(rrij) as the fraction of macroscopic states in which 
the cavity magnetization of vertex j takes the value nij, and denote "Pj,k,...{ m ji m ki ■ ■ •) as the fraction of macroscopic 
states in which the cavity magnetization of vertex j,k, . . . take the value rrij, m^, . . ., respectively. We extend the 
Bethe-Peierls approximation Eq. (fl~5|) to the level of macroscopic states and assume that 

Vj,k,...{ m ji m k, ■■■) = Vj(rrij)Vk(mk) ■ ■ ■ ■ (19) 

Equation (|19p is equivalent to saying that, the fluctuations (among all the macroscopic states) of the cavity magneti- 
zation of two different cavity vertices are mutually independent of each other. Due to the absence of short loops in a 
random graph, Eq. (|19p turns out to be a rather good approximation. 

To obtain a self-consistent equation for the marginal probabilities {V 3 (rrij)}, we add a new vertex i and connect it 
to the K — 1 cavity vertices of Qk(N,K — 1). The quenched coupling constant Jy of each newly added edge is set 
to be Jij = ± J with equal probability. This results in a new cavity network Gk(N +1,1) ofA^ + l vertices and one 
single cavity vertex i (see Fig. 03). In the corresponding macroscopic state a of the new cavity system Qk(N + 1,1), 
the cavity vertex i feels a cavity field hi. 

To calculate the cavity field hi in the new cavity system, we first notice that the energy difference between the 
cavity network Gk{N + 1,1) and the old cavity network Gk(N, K — 1) is 

jed'i 

where d'i denotes the set of K — 1 nearest-neighbors of vertex i in the cavity graph Gk{N + 1, 1). In the macroscopic 
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state a, the partition function for the cavity system Gk{N + 1,1) is 

Z a (G K (N+ 1,1)) =J2 E e^^W- 1 ')-^ 1 

fi {(Ti,...,CTjv}eC( 

^ c -/3F a (p K (N,K-l)) ^£jem 

(Tj : jed'i 



-0F a (g K {N,K-i)) j-j- 

-^(ffifCiV.K-l)) J-j- 



cosh (3 J \- 



jed'i 



cosh (3u(Jij, hj) 

cosh pjij 
cosh 0u(Jij , hj) 



22 exv(f3hi(Ji) 
(2 cosh f3hi) . 



(21) 

(22) 

(23) 
(24) 



In going from Eq. (|2"Tj) to Eq. ([2"2]). we have used the Bethe-Peierls approximation Eq. fig)), F a {Q K {N,K - 1)) is 
the free energy of the cavity system Qk(N,K — 1) in its macroscopic state a. In Eq. (|2"3"|) , the quantity u(Jij,hj) is 
defined as 

u(Jij, hj) = — atanh(tanh/3/i_y tanh/3Jy) ; (25) 
P 

and the quantity /i,; is calculated according to 

hi= J2 u(Jij,hj) . (26) 

jed'i 

From Eq. (|23|) we know that /ii as expressed by Eq. (|26|) is just the cavity field felt by vertex i in the cavity network 

g K (N + 1,1). 

If we know all the cavity fields on the nearest-neighbors of vertex i, then we obtain the cavity field on vertex i 
through the iterative equation (I26p . Consequently, the magnetization of the cavity vertex i in the macroscopic state 
a of the cavity system Gk{N + 1,1) is calculated as 



TO,- 



tanh fihi 



(27) 



where is a shorthand notation for tanh/3Jy, i.e., 

Wjj = tanh/3Jy . (28) 

Equation (|27p is an iterative equation for the cavity magnetization m, within one macroscopic state a. As we have 
emphasized in Sec. IIV Al the input cavity magnetization rrij in Eq. (|27[) may be different in different macroscopic 
states. As a consequence, the cavity magnetization m, of vertex i does not necessarily take the same value in different 
macroscopic states. On the contrary, its value may fluctuate a lot among different macroscopic states of the new 
cavity system Gk{N + 1,1). The task is to obtain an expression for the marginal distribution Vi(rrii) of m, among 
different macroscopic states. 

From Eq. (|24|) . we know that, after the addition of the vertex i, the free energy difference AFi between the 
macroscopic state a of the system Gk(N + 1,1) and that of the system Ok(N, K — 1) is 

cosh pjij 



cosh (3u(Jij , hj 



In (2 cosh/3/ij 

P 



= in ( i - 4) - ■§ in ( n t 1 + v ^ m i\ + n t 1 % TO i] 



(29) 
(30) 



According to the free energy Boltzmann distribution Eq. ([9]), each macroscopic state is weighted by the Boltzmann 
factor exp(— yF a ). After the addition of vertex i, the total partition function of the system Gk(N + 1, 1) is 



Z(g K (N + 1,1)) = ^Z ct (fe(^,^-l))cxp(-yAF 1 ) 

a 

= £ Z a (G K (N, K - l))j J] [ j dmjVj(mj) 

- a -I jed'i ^ 



exp(-yAFi) 



(31) 
(32) 
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We have used the factorization approximation Eq. (fTi?)) in going from Eq. (|3~lj) to Eq. (f3"2"| . Similarly, the total weight 
of those macroscopic states of the system 5k (A + 1,1) in which the cavity magnetization of vertex i being equal to 
nii is expressed as 



Y,Z a (g K (N,K-l)) 



n 

jed'i 



&mjVj(mj) 



exp(-yAFi)5 mi 



n [i+vijmj]- n [i-vijmj] 

jEd'i jEd'i 

I] [1 + + I! [l-v tj mj\ 



jed'i 



jed'i 



(33) 



From Eq. (|32)) and Eq. ([35)) we realize that, in the new system, the fraction of macroscopic states in which vertex % 
bearing a cavity magnetization m, is equal to 



Z(g K (N + l,l); mi ) 

%{ i) - WW +i,i)) ".y, 



&m j Vj{m j ) 



n [i + f tJ mj] - n [i - % m j] ■ 

jEd'i jEd'i 
jEd'i jEd'i 



(34) 



Equation (|34[) is a self-consistent iterative equation for the cavity distributions {Vi}. A steady state solution of 
Eq. (f3"4"|) can be obtained by population dynamics [1, An array of Af probability distributions Vi are stored. At 
each step of the population dynamics, K — 1 probability distributions are randomly chosen from this array of A/" stored 
distributions, and a new probability distribution is generated by using Eq. (|34[) . This new distribution then replaces 
a randomly chosen old probability distribution in the array. This iteration process is repeated many times until the 
population dynamics reaches a steady state. 



V. GRAND FREE ENERGY DENSITY 



The free energy of a macroscopic state a is defined formally by Eq. |T]). As we noted before, this expression is not 
directly applicable since we do not know what are the microscopic configurations of state a. The cavity approach 
[H circumvents this problem by calculating the grand free energy difference between a system of N vertices and an 
enlarged system oi N + 2 vertices. As demonstrated in Fig. [31 a random network 5k (A) can be constructed from a 
random cavity network 5k (A, 2(K — 1)) by adding K — 1 new edges. The grand free energy difference between these 
two systems can be calculated. Similarly, one can add two new vertices and 2K — 1 new edges to change the same 
random cavity network 5k(A, 2(K — 1)) into a random regular network (A + 2). The grand free energy difference 
between these two systems can also be calculated. From the two grand free energy differences, one can obtain the 
grand free energy density <?(/?; y) for a random regular system 5k (A). The mean free energy density of a macroscopic 
state and the complexity of the system can then be obtained from g((3;y). 




FIG. 3: The cavity approach to the spin-glass model Eq. on a random regular graph of connectivity K — 6. (A): part of a 
cavity graph with 2(K — 1) cavity vertices. (B) and (C): construction of a random regular graph of size N (B) and N + 2 (C) 
from the cavity graph with N vertices and 2(K — 1) cavity vertices. 
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A. From the cavity network Qk[N ,2(K — 1)) to the regular network Qk{N) 

The difference AH 2 between the conhgurational energy of the system on the random graph Qk (N) in Fig. [3j3 and 
that of the system on the random cavity graph Qx(N, 2(K — 1)) in Fig. is 

AH 2 =- ■/..rr.T, , (35) 

(jj')£ei 

where e\ denotes the set of K — 1 newly added edges in going from g K (N,2(K - 1)) to g K {N). Each edge in set 
ex connects two cavity vertices of Qk(N,2(K — 1)). Following the analytical procedure of Sec. HVB"1 we know that 
difference between the free energy of a macroscopic state a of the system Gk{N) and that of the same macroscopic 
state a of the cavity system Gk(N, 2(K — 1)) is 

AF 2 = AFih3) > ( 36 ) 

(i j)6ei 

where 

AP(i ' j) = Yp ln(l ~ V ^ ~ \ ln(l + v « mim i) ( 3? ) 

can be understood as the free energy increase caused by the addition of an edge (with coupling Jy) between two 
cavity vertices i and j. 

The grand free energy G(N, (3; y) of the new system Qk(N) is related to the grand free energy G( cv ) (TV, /3; y) of the 
old cavity system Gk(N, 2{K — 1)) by the following equation 

G(N,p;y)=G^(N,fty) + £ AG^ , (38) 

(ij')Gei 

where 

AG (lj) =-iln J dm, J dm^im^P^m^exp^yAF^^) (39) 
is the increase to the grand free energy caused by adding an edge (i, j). 



B. From the cavity network Qk (N, 2(K — 1)) to the regular network Gk{N + 2) 

The random network Qk(N + 2) in Fig. [3p is constructed from the random cavity network Gk(N 7 2(K — 1)) 
of Fig. (3J\ by adding two vertices (io and jo). Vertex i is connected to K — 1 cavity vertices (ji, . . . , ]k~i) of 
Gk(N, 2{K — 1)), and vertex jo is connected to the remaining K — 1 cavity vertices (ii, . . . , ik-i) of Gk{N, 2{K — 1)). 
Vertex io and jo are directly connected by a new edge (io , jo ) , so that every vertex in the graph Qk (N + 2) has degree 
K. The energy difference between the system Qk(N + 2) and the cavity system Qx(N, 2(K — 1)) is 

K-l K-l 

&H 3 = -Ji o3a <Ji (Jj - ^ J io0s a io u 3s - E J Joi s (7 Jo f7 «a • ( 40 ) 
s=l s=l 

The increase AF3 in the free energy of macroscopic state a due to the addition of two new vertices and 2K — 1 new 
edges can be obtained following the same procedure as given in Sec. IIVBI We find that 

AF 3 = AF M + AF {3o) - AF { ' loM . (41) 

In Eq. gj), AF^ is the free energy increase caused by adding vertex io and connecting it to the set dio = 
{jo, ji, ■ ■ ■ , Jk-i\ of K vertices. The explicit expression for AF^ is 

af( * o) =^ e M^y-Wn n [i-w*]) • (42) 

1 jedi M K jedi jedi J 
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The expression for AF^ ^ has the same form as Eq. (|4"!?)) . Let us emphasize that, in Eq. (|4"2|) , m,j is the cavity 
magnetization of vertex jo when vertex %q is not added, i.e., 

11 [i + Vioj.mj,]- II [i-Vioi.mj,] 

n [ i + v i js m js] + n I 1 'v.-,.;, "'.i, I 

j B edi \jo j s edi \jo 

II [ 1 + V his m is] ~ II [1- Vj ais m is ] 

m . _ *«£ a JoVo i 3 £dj \io iAA) 

II [i+Vjoi.^ia] + II [! - %oi<,"Vl 

i B £djo\io i s &dj \io 

The term AF( ioJ ' ' of Eq. (gTJ) is the free energy increase caused by setting up an edge between vertex ?o an d vertex 
jo; its expression is given by Eq. (|37p . with nii and rrij being determined by Eq. (|43p and Eq. (|44|) . respectively. 
The free energy increase Eq. (|4"Tj) can be intuitively understood as follows: Since the contribution of the edge (io,jo) 
is counted twice in AF^ and AF^°\ the free energy increase should be corrected with an edge term. 

With these preparations, we can calculate the total grand free energy G(N + 2,(3; y) of the system Qk{N + 2). 
Similar to Eq. ([38]) , we find that 

G(N + 2, /3; y) = G (cv) {N, (3; y) + AG {io) + AG (jo) - AG {i ° Jo) . (45) 

In Eq. (|4"5)) . AG^ -* is the increase to the grand free energy caused by adding vertex i$ and connecting it to K vertices, 
with 



AG (io) = [ Yl ^mj^imjJexpi-yAF^)) 
y V j B edi J 



(46) 



where Vj {mj Q ) is the cavity magnetization distribution of vertex jo m the absence of vertex AG^ 0J '°^ in Eq. (|45|) 
is calculated through Eq. using Vi and Vj . 

C. Averaging over the quenched randomness 

The grand free energy density of the spin-glass system is 

m \ y G(N + 2,P;y)-G(N,(3;y) 
9{P; y) = Jim z ■ (47) 

An explicit expression for g{(3;y) can be written down by applying Eq. ()38j) and Eq. (|45p . which is a function of the 
quenched randomness in the system. The grand free energy density has the nice property of self-averaging [3j, namely 
the value of g([3; y) as calculated for a typical system is equal to the averaged value of g{(3; y) over many systems with 
different realizations of the quenched randomness in the graph connection pattern and in the edge coupling constants. 
When the quenched randomness is averaged out, we obtain that 

g(P;y) = AG^-^AG^ , (48) 

where AG^ and AG^'^ are calculated through Eq. |46|) and Eq. (|39|) . respectively; and an overline indicates averaging 
over the quenched randomness of the spin-glass system. 

The mean free energy density of a macroscopic state of the system is related to g(f3; y) by 

f(a \ dyg{(3; y) dg{(3;y) 

and the complexity of the system at given value of the reweighting parameter y is 

E(P;y)^y(f- 9 (/3;y))=y^^M. (50) 
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VI. THERMODYNAMIC QUANTITIES FOR A SINGLE INSTANCE OF THE SYSTEM 

The discussion in Sec. fVl was concerned with the typical properties of an ensemble of spin-glass systems governed 
by a given distribution of quenched randomness. One important advantage of the cavity approach is that, for a single 
instance of the quenched randomness, the thermodynamic properties can also be calculated. Under the Bethe-Peierls 
approximation, the total grand free energy of a spin-glass system Eq. @ on a graph Q with couplings {Jij} is equal 
to 

G {Ji . }>s (/3;y) = ^AGW. })g (/3;y)- £ AGg> }e (/3; y) . (51) 

It is easy to see that the above equation is consistent with Eq. (|18")) . In Eq. (fS"Tj) . AG^j.j g denotes the contribution 
of vertex i and its associated edges to the total grand free energy of the system; its expression has the same form as 
Eq. " 



AGf Jt]} g {(3; y) = ~ lnM [ | / dm,- ^ ^m, ti ) e^**™ ) , (52) 
Seat 

where mj^i is the cavity magnetization of vertex j (with respect to vertex i) in a macroscopic state a, and Vj^iirrij^i) 
is the probability distribution of this cavity magnetization among all the macroscopic states; AF^ 1 ' is the free energy 
contribution (in a given macroscopic state) of vertex i and its associated edges, which has the same form as Eq. (|42|) 
but with rrij replaced by ro,_>i. Similarly, the term AGrj?, g in Eq. (|51|) is the contribution of an edge to the 
total grand free energy of the system; it is expressed as 

AG {ji},g(fcy) = ~^ ln (/ dTO ^ / ^^V^im^r^jim^e-y^ 13 ^ , (53) 

where AF^' is given by Eq. (|37[) but with m, replaced by and replaced by TOj-_>j. 

The set of 2M (M being the total number of edges in the graph Q) probability distributions Vj^i(mj^i) in Eq. (|5Tj) 
should be carefully chosen such that the total grand free energy achieves a minimal value at given (/3, y) values. In 
other words, for any edge of the graph Q, the variation of Gij^y g with respect to both Pj^i and Pi-tj should 
vanish: 

spj^i sr, ., ■ v ' 

Equation (|5^|) results in the following self-consistent Bethe-Peierls equation for the P^j's: 



-Mm/' 



(0 



7', .,(///, ./i x Y[ J dm k ^iPk^i(m k ^i) 



/ EI I 1 + Vi k m k ^i\ - J] [1 - v ik m k ^i] . 
e-y^slm^j-^j- ; ^ T | , (55) 



, n [i + Vik-nik^} + n [i - v ik m k ^i] 

kedl \3 x kedi\j k£di\j 



where 

Ap('»= 

2/3 ^ v /? ■ x 

k£di\j kGdi\j k£di\j 

AF^ has the same physical meaning as AF± of Sec. HVBl Equation (|56p is consistent with Eq. (f3"l|) of Sec. II V Bl 
Similar to Eq. and Eq. (|50[) . the mean free energy of a macroscopic state of the sample is expressed as 

F {Jijhg (P;y) = G {Ji . hg (^y)+y dG{J ^ if3;y) , (57) 



and the complexity of the system is calculated through 



^{j^Mv) = jjv Q- y ■ (58) 
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Because of Eq. (j5"4")) , the first derivative of G{j t .y t g with respect to y can easily be expressed. We find that Eq. ([5 
can be re-written as 



^}, e (/3;2/) = E< AFW >- E ( AFit3) ) 



(59) 



teg 



where 



(AF (ij) ) 



jedi L 



II /dm 



-p. . 



( m J 



J dm^j J d;;i, ./P; .,(,»>, ./.in .,!»;, ' A/- 

J dm^, / dm i ^ rj Vj^i{rn j ^ i )'Pi^j{rni^ rj )e-y AF{z3) 



(60) 



(61) 



The mean free energy of a macroscopic state is also decomposed into vertex contributions and edge contributions. 

According to the discussion in Sec. IIII1 in the mean-field theory, the reweighting parameter y should be chosen 
such that its value is closest to the physical inverse temperature (3. Therefore, at a given value of (3, the mean free 
energy F^j \ t g((3) of the system is obtained by setting y — (3 in Eq. ([57]) or Eq. (|59"|) , provided that the complexity 
^8,{Ji }(0Vy) is always non-negative in the range of < y < (3. If, on the other hand, ^•g,{j i -}{0'iV) changes from 
being positive to being negative at a point of y — y* < (3, we should set y to y — y* . In this later situation, the mean 
free energy of the system is equal to the maximal value of the grand free energy, Gg,{j 4 •}(/?; J/*)- After the mean free 
energy F^j..^g{j3) of the spin-glass system is obtained, the mean energy of a macroscopic state is calculated through 



E {Ji . h g(f3) = F {Jij}>g (p) + (3 



d/3 



and the mean entropy of a macroscopic state is calculated through 



2 dF { j ij} ,g(P) 



d/3 



(62) 



(63) 



Ensemble-averaged densities of mean energy and mean entropy can also be obtained from Eqs. ([62]) and (|63|) 
by averaging out the quenched randomness. An alternative derivation for the mean energy density is given in the 
appendix. 



VII. NUMERICAL RESULTS FOR THE ±J MODEL WITH K = 6 



We have applied the mean- field treatments of Sec. IIVI and Sec. [V] to the ±J spin-glass model Eq. (01 on a random 
regular graph of vertex degree K — 6. (In what follows, energy is in units of J.) At temperature T = 0.8 (i.e., 
f3 = 1.25), Carrus, Marinari and Zuliani (as cited in Ref. |4j) estimated that the energy density of this system is 
e = —1.799 ± 0.001. The mean-field theory of Ref. [3| also predicted the same result. This reference also predicted a 
mean-free energy density of / = —1.858 ± 0.002. 

In the present population dynamics simulation, each probability profile Vi(mi) in Eq. (|34|) was represented by an 
vector of m = 200 elements. A total number of Af such probability profiles were stored in the computer and they 
were updated using Eq. (|34[) . In our work, various M values ranging from 32 to 6, 000 were used. In each elementary 
update of the probability distribution Vi{mi), a Metropolis importance sampling [24] with inverse temperature y was 
exploited to generate a total number of n x m magnetizations m t . These magnetizations were stored in an ordered 
set. The averaged value of the first (second, third, . . .) n magnetizations in this set is assigned to the first (second, 
third,. . .) element of the new Vi- We chose n = 200 in our simulations. 

The grand free energy density g(/3; y) of the system as calculated according to Eq. (|48)l is shown in Fig. |4j The 
grand free energy density g(y) first increases with the reweighting parameter y till y reaches y* — 0.30 ± 0.01, at 
which point g{/3;y) attains a maximal value of g = —1.8590 ± 0.0001. This maximal value of g(f3;y) corresponds 
to the best estimate of the mean free energy density of the system by the present mean-field method. It is in close 
agreement with the prediction of Ref. [4| . We have checked that the estimated mean free energy density value is not 
sensitive to the population size M (see Fig. [S]). We have also calculated the mean energy density and mean entropy 
density for the model system at j3 = 1.25 using a population size of M = 4, 000, with the mean energy density being 
e = —1.8007 ± 0.0002 and the mean entropy density being s — 0.0732 ± 0.0005 (see Fig. [S]). These predictions are 
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FIG. 4: The grand free energy density g((3;y) of the ±J spin-glass model on a random regular graph of vertex degree K = 6. 
The inverse temperature is fixed to j3 = 1.25. Symbols are results obtained by population dynamics simulations using a 
population size of TV = 256. The dashed line is a fit to the data using g(y) — a — b(y — y*) 2 , with the fitting parameters 
a = -1.8590 ± 0.0001 and y* = 0.30 ± 0.01. 
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FIG. 5: Mean free energy density of the ±J spin-glass model on a random regular graph of vertex degree K — 6 at f3 = 1.25 
as obtained by population dynamics with different population size M. (Inset) the optimal reweighting parameter y* . 

also in good agreement with the numerical results of Ref. [ij and with the simulational work of Carrus, Marinari 
and Zuliani. These good agreements of the present theoretical results with earlier simulational and numerical work 
suggest that the present cavity approach based on the concept of cyclic heating and annealing is feasible. 



VIII. CONCLUSION AND DISCUSSION 



As a summary, in this paper we have calculated the thermodynamic properties of a spin-glass model by combining 
the physical idea of repeated heating-annealing and the cavity approach of Mezard and Parisi [3] • We have assumed 
that, during a cyclic annealing experiment, all the thermodynamic (or macroscopic) states of the spin- glass system 
at a given low temperature T will be reachable, but with different frequencies which decrease exponentially with 
the free energy values of the macroscopic states. By using this free energy Boltzmann distribution and by using 
the Bethc-Pcicrls approximation, the grand free energy of the spin-glass system can be calculated as a function of a 
reweighting parameter y; and from the knowledge of the grand free energy, the mean free energy, energy, and entropy 
of a macroscopic state of the system can also be obtained. For the ± J spin-glass model on a random regular graph of 
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FIG. 6: The energy density, free energy density, entropy density, and complexity as a function of the reweighting parameter y 
for the ±J spin-glass model on a random regular graph of vertex degree K — 6 at inverse temperature /3 = 1.25. Energy unit 
is J. The symbols represent numerical population dynamics results using N = 4000. The dashed line in the bottom panel is a 
fitting to the numerical data with E = ao + ai/y + 02/ 'y' 2 + 0,3 /y 3 in the range of 0.2/3 < y < 0.3/3. To reduce computational 
time, for the simulations reported in this figure, a faster but less precise numerical recipe was used to generate a new probability 
profile Vi(uii) using Eq. (|34f) . This might be the reason for the relatively large fluctuations of the mean densities with the 
reweighting parameter y. 



vertex degree K = 6, the theoretical predictions of the present work are in good agreement with the results of earlier 
simulational and numerical calculations. 

For the ±J model at f3 — 1.25, we found that the complexity E becomes negative before y reaches (3. Therefore, the 
thermodynamic properties of the system are contributed by those macroscopic states which have the global minimal 
free energy density. However, it may exist other systems for which the complexity is still positive even when the 
reweighting parameter y reaches [3. For such systems, the thermodynamic properties are not determined by those 
macroscopic states of the minimal free energy density, but by a set of 'metastable' macroscopic states. On the one 
hand, these metastable macroscopic states have a higher free energy density; on the other hand, the number of such 
macroscopic states greatly exceeds the number of macroscopic states of the global minimal free energy density. In 
the competition between these two factors, the 'metastable' macroscopic states may win. Then the system will reach 
one of these metastable macroscopic states almost surely in each round of the temperature annealing experiment. We 
will work on a model system with many-body interactions to check whether this is really the case. 

As in the work of Ref. the present theoretical treatment also assumes that the configurational space of the 
spin-glass system breaks into exponentially many macroscopic states, but there are no further organizations of these 
macroscopic states; and it is assumed that each macroscopic state is ergodic. As pointed out in Ref. [25j, this first- 
step replica-symmetry-broken (1RSB) cavity solution might be unstable. Two types of instability is conceivable. 
The type I instability concerns with possible clustering of macroscopic states into 'super-macroscopic' states; the 
type II instability is caused by splitting of each macroscopic state into many 'sub-macroscopic' states. To account 
for the further clustering of macroscopic states appears to be relatively easy in the present mean-field framework: 
for each cluster of macroscopic states, a vertex has a probability profile concerning its cavity magnetization; this 
probability profile is different in different clusters, and we can introduce a probability distribution of probability 
profiles to characterize this variation [H. Zhou, submitted to Comm. Theor. Phys. (Beijing)]. On the hand, to take 
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into consideration the possibility of splitting of a macroscopic state, maybe one has to introduce other reweighting 
parameters in the mean- field theory. This is an important issue waiting for further explorations. 

Another important issue is related to the updating of the probability profiles through the iterative equation ([34]) or 
(|55|) . In the present paper, we used Metropolis importance sampling technique to get an updated probability profile 
of cavity magnetization. This method appears to be rather precise but time-consuming. If faster algorithms with 
comparable precision could be constructed, it will be highly desirable. 
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Appendix: On an explicit expression for the mean energy density of a spin-glass system 

In the main text we did not give an explicit formula for the mean energy density e of a spin-glass system. Instead, 
the mean energy density was calculated from first knowing the mean free energy density of the system [see, e.g., 
Eq. (|62[) ]. In this appendix, we derive an explicit expression for the mean energy density e. The assumptions used in 
this derivation will be clearly pointed out. 



A. Energy contribution of adding K — 1 edges 



Consider again the two systems, Gk\N, 2(K — 1)) in Fig. [3]A_ and Qk{N) in Fig. [3)3. (For notational simplicity, let 
us hereafter referr to these two systems as system 'a' and 'b', respectively.) As was mentioned in Sec. IV Al for a given 
configuration the energy difference between these two systems is AH 2 . When averaged over all macroscopic states, 
the mean total energy of the system 'a' is 

«"">•> = »;(-"'•«") _ (64) 

where F£ is the total free energy of macroscopic state a of system 'a', and H a (<j) is the the configurational energy of 
system 'a'. Similarly, the mean total energy of the system 'b' is 

V- ( T?a AE1 ^E 5ea [ff Q (5 ! )+AH2]c X p(- / 3H»(ff)- / 3AH 2 ) 

> exp — - yAF 2 — H t 

«**>«> = - V , E K . : (-^rz ) (65) 

N ' J2exp(~yFS-yAF 2 ) V ' 



where AF 2 is defined by Eq. (|36|) . The difference of these two mean energy expressions is 

{{H b ) a )-{{H a ) a )=V 1 + V 2 . (66) 



In the above equation, 



where 



1 j:exp(-yFS-yAF 2 ) 

a 

= £ (AEM) , (68) 

(«ii)eei 



M1 , f^J^P^V^e-^^^ 



/ dm, / dm j 7 7 i(m i )P i (m J -)e-vAF(«) 
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In going from Eq. (|67p to Eq. (|6"5|) we have used the Bethe-Peierls approximation both at the level of microscopic 
configurations [Eq. (fTS|) ] and at the level of macroscopic states [Eq. (fl9]) ]. 
The term V2 in Eq. (|66|) is equal to 



E exp - j/AF 2 ) — |; 7 s y E exp(-yF«) — |; , ) , 

2 " Eexp(-2/F«-yAF 2 ) £exp(-^«) 

a a 

At this point, let us introduce two assumptions: 

(i) The distribution of the configurational energy of the cavity system 'a' and that of the energy increase can be 
regarded as mutually independent within a macroscopic state. 

(ii) The distributions of the free energy of the cavity system 'a' and that of the free energy increase can also be 
regarded as mutually independent. 

If these two assumptions hold true, then we have 

V 2 = . (71) 



B. Energy contribution of adding two vertices and 2K — 1 edges 



Now consider the two systems, Gk[N, 2(K — 1)) in Fig. [3]A (system 'a') and Gk(N + 2) in Fig. [3p (system 'c'). 
Following the same analytical procedure as given in the preceding subsection, we find that, if the two assumptions (i) 
and (ii) listed above are valid when going from system 'a' to system V, then the mean total energy of system 'c' is 
related to that of system 'a' by the following simple equation 

((H c ) a )-((H a ) a ) = (AE^'^)+ Yj (A£ ( ' ioJs) ) + (A£ (J043) ) . (72) 

js£di \jo i s £dj \io 

(Notice that in calculating (AE^ ^^ through Eq. (|69[) . m.; should be the cavity magnetization of vertex i$ in the 
absence of vertex j s .). 

Combining Eq. (f6"6")) and Eq. (f72")) . we finally obtain that the mean free energy density of the system is 



e(/3;y) = y <A£fe)) . (73) 

The same analytical procedure can be applied to calculate the mean free energy density. Under the assumption (ii) 
of the preceding subsection, we obtain that the mean free energy density / is equal to 



my) = (AF(i))--(AF(ij)) , (74) 

which is just the same as Eq. (|5§j) . 

Based on the assumption of exponentially many macroscopic states, it was argued in Ref. Q that, the assumption 
(ii) of the preceding subsection should be valid under the Bethe-Peierls approximation. In other words, the joint 
distribution of the free energy of the old cavity system F and the free energy increase AF is factorized. The 
consistence between Eq. (jT4"]l and Eq. ([59)1 also confirmed this point. Since each macroscopic state ifself contains an 
exponential number of microscopic configurations, we believe that the assumption (i) is also valid in the Bethe-Peierls 
mean-field framework. The mean energy density of the ±J spin-glass model as shown in Fig. [6] was calculated using 
Eq. (|73p . and the output was in good agreement with earlier simulation result. Maybe one can further check this point 
by working on some simple models which are exactly solvable both analytically and computationally. The maximum 
matching problem as studied in Refs. [2|| [27j could be a good candidate model system. 
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